System and method to determine slide quality of a digitized microscope slide

ABSTRACT

Methods, media, and systems for assessing the quality of a digital image. In an embodiment, both a micro-analysis and macro-analysis are performed. The micro-analysis comprises dividing the digital image into a plurality of blocks, for two or more of the plurality of blocks, determining a score based on a spatial frequency of the block, and generating a score map for the digital image based on the score for each of the two or more blocks. The macro-analysis comprises detecting artifacts in the digital image, computing a degradation score based on detected artifacts, and computing a whole-slide-quality score based on the score map and the degradation score.

RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 14/060,534, filed on Oct. 22, 2013, which is a continuation ofU.S. patent application Ser. No. 13/515,073, filed on Jun. 11, 2012 andissued as U.S. Pat. No. 8,565,503 on Oct. 22, 2013, which is the U.S.national stage of International App. No. PCT/US2011/039173, filed onJun. 3, 2011, which claims priority to U.S. Provisional Patent App. No.61/396,971, filed on Jun. 4, 2010, all of which are hereby incorporatedherein by reference in their entireties.

BACKGROUND

1. Field of the Invention

The present invention relates to the field of digital pathology and moreparticularly relates to the assessment of image quality based oncomplexity and spatial frequencies and the presentation of saidassessment using both visual and quantitative results.

2. Related Art

The cornerstone of pathology is the glass microscope slide with itsbiologic specimen. Traditionally, a pathologist used a conventionalmicroscope to inspect the specimen. The pathologist would navigate thefield of view by moving the slide underneath the microscope's lens andincrease or decrease the field of view by selecting lenses of differentmagnifications. To adjust for the variable thickness of a specimen, thepathologist would adjust the focus by moving the optics up and down andmodify the brightness by adjusting the light aperture. In this manner,the pathologist interactively adjusts for acceptable image quality.

Similarly, the cornerstone of digital pathology is the digitizedmicroscope slide (“digital slide”), an image file of the entire slide.Digital pathology scans the microscope glass slide at a highmagnification, automating the pathologist's actions of focus and dynamicrange adjustment as it captures the image and stores the digital slide.The pathologist inspects the digital slide using viewing software. It iscritical that the image is scanned without fault because the viewingsoftware simply displays the captured digital slide and cannot re-focusthe image or offer dynamic range adjustments. Common problems thatplague scanning software include, but are not limited to, out of focusscans and the introduction of scan-hardware related artifacts intoimages.

Manually reviewing each image for sufficient scan quality is timeconsuming because a given digital slide image may be very large (e.g.,as large as 200K×100K pixels). Additionally, in many cases only anexpert may be able to properly judge variations in the quality of adigital slide and these judgments are highly subjective. For example,scan artifacts can make distinct sub-cellular structures apparent in oneslide region but difficult to distinguish in a nearby region. Scanartifacts can also change tissue architecture from a crisply patternedtexture to a smooth plane. Furthermore, even a properly scanned digitalslide may lack sufficient quality for proper analysis and review.Accordingly, there exists a need for a system that is capable ofmeasuring the quality of the digital slide image and identifyingscan-related artifacts.

SUMMARY

Determination of image quality is a difficult task for a computersystem, whereas an expert may readily distinguish between a good qualityimage and a bad quality image. Particularly, an expert may judge thecontent and context of an image and see differences in focus andcontrast. However, research shows that no single metric stands out asthe dominant indicator of image quality for a wide variety of imagery.Instead, combinations of metrics in classification schemes provethemselves more reliable indicators yet have not been proven to operatewell in digital pathology.

In digital pathology, the imagery is limited to biologic specimens. Thelook of a specimen is complex and its nature varies from patient topatient. However, areas of interest in most specimens are generally“busy” areas of activity with respect to tissue and cell structures, asillustrated in FIG. 6. These “busy” areas of activity translatemathematically into the presence of spatial frequencies. The digitalslide quality determination system and method use the spatialfrequencies that are present in a high quality pathology image. Bydetermining the presence of certain spatial frequencies in the image,the system generates a quantifiable assessment of the digital slideimage quality.

By analyzing the images based on complexity and spatial frequencies, anexample embodiment described herein provides visual feedback on thequality of the whole slide by overlaying the image in the viewingsoftware with a color coded “heat map” of local area quality. Thus, theuser is presented with both an absolute quality measure for the wholeimage and the ability to see the quality variability of local areas at aglance.

Other features and advantages of the present invention will become morereadily apparent to those of ordinary skill in the art after reviewingthe following detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The structure and operation of the present invention will be understoodfrom a review of the following detailed description and the accompanyingdrawings in which like reference numerals refer to like parts and inwhich:

FIG. 1 is a schematic flow diagram illustrating one embodiment of amicro-analysis process;

FIG. 2A is an example embodiment of 8×8 frequency transforms applied toa 32×32 pixel image block;

FIG. 2B is an example embodiment of 16×16 frequency transforms appliedto 32×32 pixel image block;

FIG. 3 is a schematic flow diagram illustrating one embodiment of amethod of a macro-analysis process;

FIG. 4A is a block diagram illustrating an example embodiment of asystem for determining the quality of a digital microscope slide;

FIG. 4B is a block diagram illustrating example modules found in ascanning system;

FIG. 5A is a block diagram of a first embodiment of an opticalmicroscopy system according to the present invention;

FIG. 5B is a block diagram of a second embodiment of an opticalmicroscopy system according to the present invention;

FIG. 5C is a block diagram of a third embodiment of an opticalmicroscopy system according to the present invention;

FIG. 6 illustrates an area of interest in a digital slide;

FIG. 7 illustrates an example embodiment of a digital slide;

FIG. 8 illustrates an example embodiment of a markup image of colorcoded image block quality score;

FIG. 9 illustrates an example of a high resolution cut out of a markupimage of a low quality area;

FIG. 10 illustrates an example of a high resolution cut out of a markupimage of a high quality area;

FIG. 11 illustrates an example of a high resolution cut out of themarkup image displaying an edge effect from scanning;

FIG. 12 illustrates an example of bounding box outlines around the edgeof area artifacts detected on a sample scan;

FIG. 13 is a block diagram illustrating an example computer system thatmay be used in connection with various embodiments described herein; and

FIG. 14 illustrates examples of different variations in digital slidecontent.

DETAILED DESCRIPTION

Certain embodiments disclosed herein provide systems and methods forassessment of image quality based on complexity and spatial frequenciesand the presentation of said assessment using both visual andquantitative results. For example, one method disclosed herein allowsfor analyzing digital slide images based on complexity and spatialfrequencies to provide visual feedback on the quality of the whole slideby overlaying the image in the viewing software with a color coded “heatmap” of local area quality and thereby provide the user with both anabsolute quality measure for the whole image and the ability to see thequality variability of local areas at a glance. After reading thisdescription it will become apparent to one skilled in the art how toimplement the invention in various alternative embodiments andalternative applications. However, although various embodiments of thepresent invention will be described herein, it is understood that theseembodiments are presented by way of example only, and not limitation. Assuch, this detailed description of various alternative embodimentsshould not be construed to limit the scope or breadth of the presentinvention as set forth in the appended claims.

Since digital slide content is diverse, as illustrated in FIG. 7,measuring the spatial frequencies of the whole image would notcharacterize the image quality well. Pathology imagery containsconsiderable white space, fragmented specimens of no import, and coverslip edges and air bubbles. An expert reviewing a given imageintuitively ignores these issues and does not consider them material tothe image quality. Therefore, the present system similarly discountsthem.

FIG. 14 illustrates examples of different variations in digital slidecontent that result from protocol differences, stain kit selection, andpathologist staining preferences. Biologic samples present extraordinarydifferences between patients, pathology and organs resulting in imagevariations. Slide preparation also varies the digital slide content byproducing tissue folds, adding foreign matter, microtome blade chatterand different sample thicknesses.

The digital slide capture process is conducted at a wide range ofresolutions. Additionally the digital slide capture process may employvarious image compression techniques (e.g. JPEG, JPEG2000), each ofwhich may have settings uniquely tailored to establish a desiredcompression quality.

In order to tolerate the variability of a given specimen, the exampleembodiment of the slide quality determination system executes a two-partanalysis: a micro-analysis process followed by a macro-analysis process.

Micro-Analysis

The micro-analysis process evaluates the whole image by breaking it downinto small parts (referred to herein as “regions or blocks”) andperforming quality analysis on each of those parts. FIG. 1 illustratesan example embodiment of a micro-analysis process 100 performed by aslide analysis system in accordance with the present invention.

Micro-Analysis process 100 begins at step 105.

At step 110, process 100 may evaluate the image as a grid of smallimages called blocks or regions. The size of a given block may bedependent upon the resolution of the image. For example, a block may belarge enough to contain one or more cells. Empirically, a block size maybe an n×n block with edge lengths in the range of 32-96 pixels, such asfor example, 32, 64, or 96 pixels. However, various embodiments mayexist with pixel squares having dimensions greater than 96 pixels orless than 32 pixels. Furthermore, the blocks are not limited to squaresbut may include various other shapes capable of making up an imageslide, including, but not limited to, rectangular blocks, stripes,polygonal regions, circular regions, etc.

At step 115, process 100 may qualify each block. The qualification stepserves to determine which blocks have proper specimen content. Forexample, a block containing too much white space may be classified as“background”, or a block that is too dark or light may be classified as“not-processed”. Further metrics may determine whether a block containssufficient edge activity, contrast, etc.

At step 120, qualified blocks may be analyzed with frequency transforms.This may include analyzing the blocks' spatial frequency content. Theanalysis may include performing a two dimensional Fast Fourier Transform(“FFT”), Discrete Fourier Transform (“DFT”), Discrete Cosine Transform(“DCT”) or Discrete Sine Transform (“DST”). The present exampleembodiment is illustrated using a DCT. However, the use of DCT in thepresent example is not intended to illustrate a preference of biastowards or against the use of DCT, or any other viable single orcombination of frequency transforms.

At step 120, a plurality of DCTs may be performed in a grid pattern overthe blocks. An overlapping pattern could also be performed in which casemodified-DCT (“MDCT”) is performed. DCTs within an image block may bereferred to as “partial-DCTs.”

In the example embodiment, the partial-DCT size may be set such that atleast 4 partial-DCTs may fit within an image block. For example, a 32×32pixel image block can use an 8×8 or 16×16 partial-DCT. FIG. 2Aillustrates an example embodiment with 8×8 partial-DCTs in a 32×32 pixelimage block. FIG. 2B illustrates a second example embodiment with a16×16 partial-DCTs in a 32×32 pixel image block.

At step 125, process 100 may combine the multiple frequency transformsinto a Block Score. Process 100 may consolidate the partial-DCTs into a“block-DCT.” The block-DCT may have the same two dimensional size of theblock. During the consolidation, partial-DCTs with values that are abovea given threshold may contribute to the corresponding block-DCT. Theamount each partial-DCT contributes to the block-DCT may be based on themagnitude of the partial-DCT. Alternatively, each partial-DCT maycontribute a preset uniform value, such that the block-DCT may become acount value for the number of partial-DCTs in the block that exceeded apredetermined threshold.

A quality score for the block may be generated from the block-DCT. Theblock-DCT may be required to meet a minimum magnitude to indicate thepresence of a frequency. For example, the ratio of partial-DCTs above athreshold forming the block-DCT may be representative of the amount offrequency content in the image block. This value may be determined bydividing the number of partial-DCTs above a threshold forming theblock-DCT by the number of partial-DCTs making up the block.Alternatively each partial-DCT may be weighted to emphasize thecontributions of one frequency range; for example, the higherfrequencies could be weighted more than the lower frequencies. Theweighted sum of the partial-DCTs of a given frequency may then benormalized by the sum of the weights to produce a value between 0 and 1.

Multiplying the value by 100 provides a score from 0 to 100. A score of70 may indicate that 70% of the possible spatial frequencies werepresent. An excellent quality image block may score in the 80's or 90's.An acceptable quality block may score in the 60's or 70's. A poorquality image block may score in the 50's or below. Table 1 illustratesan example numeric block score range that can serve to score thedifferent blocks.

TABLE 1 Block Score Score Description Markup Image Color 100-85 Excellent Blue 84-75 Good Green 74-65 Decent Yellow 64-55 Low Orange54-0  Poor Red

At step 125, process 100 may generate a markup image from the qualifiedblocks. The markup image, also known as an overlay image, may begenerated by blending each block's original image converted to grayscale with a block of color related to the block's score. Any schemecould be devised, such as, for example, the color coding schemepresented in Table 1.

FIG. 8 illustrates an example embodiment of a markup image of colorcoded image block quality scores. A benefit of the example embodiment isthat a glance at the markup image gives the reviewer an instant visualunderstanding of the whole slide quality. FIG. 9 illustrates an exampleof a high resolution cut out of a markup image of a low quality area.The red and orange areas are indications of lower quality and yellowareas are passable. FIG. 10 illustrates an example of a high resolutioncut out of a markup image of a high quality area. When the markup imageis predominantly blue and green the slide quality is very good. FIG. 11illustrates an example of a high resolution cut out of the markup imagedisplaying an edge effect from scanning. The small block sizes provideaccurate determination of image capture artifacts, shown by a sparse redline of poor quality mixed with blocks of varying quality.

At step 135, process 100 generates a score map that is compiled for useby a macro-analysis process, as exemplified below. The dimensions of thescore map are compared to the image size scaled by the image block size.Each of its pixel values may represent the block's quality score,ranging from 0 to 100 or an indication that the block was classified aswhite space or was not processed.

Process 100 ends at step 140.

Macro-Analysis

The Macro-analysis process summarizes the slide quality, the amount ofspecimen measured and the location of scan-hardware related artifacts.The macro-analysis compiles informational trends among all of thosesmall image parts to form the slide quality score. FIG. 3 illustrates anexample embodiment of a macro-analysis process 300 performed by a slideanalysis system in accordance with the present invention.

Macro-analysis process 300 begins at step 305. In the exampleembodiment, prior to beginning process 300, all image blocks areanalyzed in micro-analysis process 100. Macro-analysis process 300receives the score map, described in step 135 above. Alternatively, thesystem may function in parallel, pipelining the image blocks frommicro-analysis process 100 to macro-analysis process 300 as thenecessary blocks are processed.

At step 305, process 300 pre-processes the blocks. Pre-processing buildson block qualification step 115 from micro-analysis process 100.Previously, step 115 labeled blocks containing questionable content(e.g., the blocks included folds, saturated specimen, foreign materials)as not processed. Step 305 removes the scores on the specimen edges andfragments in the score map so that they are ignored in the subsequentsteps in macro-analysis process 300.

At step 310, process 300 determines the percentage of specimen analyzed.This percentage may serve as an indicator of specimen artifacts. Thepercentage value may also be used as a confidence measure for the finalwhole slide quality score. For example, if only 70% of the specimen wasdeemed fit to process, it might be recommended that a quality controltechnician review the digital slide and its markup image to make a finaldecision upon the slide's readiness for pathologist diagnosis. Also, anaverage score can be directly computed from the preprocessed score map.However, one limitation of this approach is that locales of poor focus(i.e., low scores) may not significantly impact an image with blockswith average values of overwhelmingly high scores. For example, theaverage quality score for the digital slide may be 71 when considered ona scale of 0 to 100. However, the average score of 71 may not reflectthe slide's poor quality with respect to specific areas.

At step 320, process 300 performs artifact detection and degradationscoring. digital pathology scanners have common and unique hardwarecapture related artifacts that they may introduce into the digitalslide. For example, a scanner may capture areas of the slide out offocus, have blurred edges along the capture field of view, or introducelight reflections or illumination fluctuations that cause photometricvariations and motion blur. Step 320 may detect these artifacts andmeasure their impact (degradation) upon the slide quality.

The following example embodiment illustrates example of two types ofartifacts (i.e. image capture and local area artifacts). However,various other artifacts may be detected and the scoring thereby adjustedappropriately.

Image Capture Artifacts: digital pathology scanners use either area orlinear cameras to receive the image in the optical field of view anddigitize it. Area cameras capture snapshots that are tiled together in amosaic format to produce the large digital slide. Linear camerascontinuously capture strips along one dimension of the image andassemble each strip adjacent to the next.

A scanner based upon an area camera can have both horizontal andvertical striping along the two dimensions of the captured tile. Thestriping can be caused by optical fall off that degrades the imagefocus, tilt between the camera's focal plane and the stage holding theglass slide, and other such optical-mechanical design issues. A scannerbased upon a linear camera will only have a striping effect along onedimension, the direction of motion.

A search for score discontinuity trends along the horizontal andvertical dimensions in the score map is performed by assembling scoreprofiles. The score profiles computed in the horizontal and verticaldirections can identify area or linear camera related edge defects.However, the score map may be further partitioned and those partitionsindependently examined for defects. For example, four vertical profilescould be created by partitioning the score map into four (4) horizontalregions. The gradient magnitude of each discontinuity in each profile ismeasured. Those with large enough magnitudes are flagged as defective.The magnitude and the proportion of the area of the image that isaffected by the discontinuity are factored into a degradation score foreach of the detected image capture related artifacts.

Local Area Artifacts: A region of the digital slide with poor quality isapparent on the markup image as a local area with a higher density ofred and orange colored blocks. Poor quality regions appear in the scoremap as areas with high densities of low scores. The areas that have apredetermined density of low score and size requirements are flagged asdefective. The region size and its density of poor scores are factoredinto a degradation score for each of the detected local area artifacts.

One additional step is taken by each artifact detection algorithm. If anartifact is detected, the affected area in the score map is labeled suchthat the subsequent artifact detection algorithm will not integrate it.

FIG. 12 illustrates an example of bounding box outlines around the edgeof area artifacts detected on a sample scan. The long, thin boxescontain image capture edge artifacts and the smaller rectangles containlocal area artifacts. It is notable that the right side of each edgeartifact outline is part of the area affected by the defect and is notillustrated in this diagram. There are many other kinds of scannerartifacts that can be identified and their severity judged. For example,motion artifacts that shift red color planes, instrument resonance, andscan velocity ripple may manifest themselves as patterns in the scoremap.

At step 325, process 300 computes the whole slide quality score. Thewhole slide quality score may be computed as the average score minuseach artifact degradation score. The whole slide quality score (“WSQscore”) ranges from 100 to 0.

At step 330, the system provides for the interpretation of the WSQscore, as determined by a study comparing an expert's quality ratingagainst that of the system. The provided comparative score may beestablished by generating a comparative scoring system based on expertrating as compared with system rating executed on the digital slide set.The interpreted results may include a summarized result set including aWSQ score, a whole slide average score, an artifact degradation score,the percentage of specimen analyzed, a markup image of color coded scoredistributions, and annotated defective artifacts.

A user of the system may choose to work with one or more these results.For example, the WSQ score may suffice to pass, fail, or visuallyinspect a slide. Alternatively, the mere presence of an artifact may beenough to require visual review.

In one embodiment, the WSQ score (or even the average score) may berequired to exceed a threshold to pass a quality control inspection. Inanother alternative embodiment, instead of using any of the threescores, a user may always want to visually interpret the markup image tofind artifacts that may not have been quantified.

For example, in one application, after computing an overall score foreach scan in the range of 1-100, the system may determine whether theslide is satisfactory or needs further inspection by comparing theoverall score to a threshold value; this provides a significant savingsin laboratory technician time since the majority of slides may provesatisfactory. If the score is lower than a given threshold, the slide isconsidered unsatisfactory and may be rescanned. In this case, the slidemay be rescanned “manually” by the lab technician in order to obtain abetter result, or unsatisfactory slides may be auto-scanned in a batch,and then only those slides that fail to rescan successfully may bemanually scanned. In further alternative embodiments, the system may tagcertain slides as neither satisfactory nor unsatisfactory, if the slidesresults fall between two thresholds, representative of satisfactory andunsatisfactory slides. These mid-range slides, representing a smallfraction of the scanned slides may be manually inspected by labtechnicians.

In application, the system may be employed in various aspects. Duringassembly of the slide scanner, the system may provide quantitative andvisual feedback as technicians assemble the slide scanner, identifyingscan artifacts that cue the technician to necessary alignment tasks.During scanner manufacturing, the processes may provide quantitativeacceptance criteria and provide quality assurance. In a field servicescenario, the system may aid in preventive maintenance checks or afterservice calls. That is, aside from operating in its pathology aimedpurpose, the system may provide a means to verify that the scanner isoperating within acceptable ranges by scanning a master glass slide andverifying quantitative and visual performance. In an operationalscenario, the system may identify locations of poor focus and instructthe scanner to automatically rescan the slide and add special focus tothose areas. In a clinical laboratory, the quantitative score may serveas an automatic quality control assessment. Also, a screening techniciancould use the combination of the quantitative score and the visual heatmap to judge the image quality.

FIG. 4A is a network diagram illustrating an example scanner system 400according to an embodiment of the invention. In the illustratedembodiment, the system 400 comprises a scanning system 440 that isconfigured with a data storage area 445. The scanning system 440 iscommunicatively coupled with a user station 430, and an image serversystem 450 via a data communication network 460. Each of the userstations 430 and image server systems 450 are configured with a datastorage area 425, 435 and 455, respectively. The data storage areas 425,435, 445 and 455 may include volatile and persistent storage includingfor example, random access memory and hard disk drives. The network 460can be any of a variety of network types including wired and wireless,public and private, or any combination of communication networks, suchas the Internet.

In operation, the scanning system 440 may digitize a plurality ofsamples to create a corresponding plurality of digital slide images thatcan be stored on the scanning system 440 or on the image server system450. The scanning system 440 may be operated directly or remotely by anoperator at the operator station 420. The digital slide images locatedat the scanning system 440 or the image server system 450 may be viewedby a user at the user station 430, where the digital image data isprovided to the user station 430 via the network 460.

While the example embodiment of scanning system 400 is presented as adistributed system linked via network 460, the system can also bearranged as a single computer system, or may include a large number ofdisparate systems for scanning and storing the digital image slides.

FIG. 4B is a block diagram illustrating an example set of modules inscanner system 440 according to an embodiment of the invention. In theillustrated embodiment, the scanner system 440 may include a tissuesample acquisition module 505, a micro-analysis module 510, amacro-analysis module 515, and a visualization module 520. In certaincombinations, the various illustrated modules collaborate to performwhole slide analysis, in accordance with previously described processes.

Tissue sample acquisition module 505 operates to obtain an initialdigital slide image from a microscope slide scanner or another source.This may include management of the scanner during scanning to handleautomatic focus, exposure control, etc. Alternatively, the tissue sampleacquisition module 505 may retrieve a digital sample slide from anavailable database.

Micro-analysis module 510 evaluates the whole image by breaking it downinto extraordinarily small parts and performing quality analysis on eachof those parts. This evaluation may be performed in accordance withmicro-analysis process 100 in FIG. 1.

Macro-analysis module 515 summarizes the slide quality, the amount ofspecimen measured, and the location of scan-hardware related artifacts.The macro-analysis compiles informational trends among all of thosesmall image parts to form the slide quality score. This process may beperformed in accordance with the macro-analysis process 300 in FIG. 3.

Visualization module 520 operates to facilitate viewing of the digitalslide image file. Image viewing adjustments such as brightness,contrast, gamma and false coloring are automatically determined usingthe stored image descriptors and the acquisition settings. In oneembodiment, viewing adjustments can be made by a user at the userstation 430 for the individual images and/or for a fused image (i.e.,the combined image of two or more individual channel images). Inaddition, when viewing a fused image, the relative translation androtation corrections may be adjusted.

Interactive image exploration tools may also be enabled by the digitalvisualization module 520 to instantly access responses on a cellularbasis. Additionally, predetermined regions of interest may containannotation that can be displayed to a user at the user station 430 toindicate meaningful biologic responses or to automaticallyquantitatively analyze responses. Additionally, the visualization andanalysis module 520 may provide a user at the user station 430 withtools to annotate regions of interest and then store such annotations inthe digital slide image file in relation to the base layer image.Advantageously, such annotations can be a useful to guide to documentartifacts in an image, regions of interest in an image, or to identify aregion of an image for reporting or quantitative analysis.

Additionally, the visualization module 520 may use predetermined orotherwise identified image features to locate similar image data orpatterns using content based image retrieval techniques. Advantageously,this utility can provide a user at the user station 30 with related caseinformation and image data.

In one embodiment, a client-server architecture permits a user at theuser station 30 to view a digital slide image located at the imageserver system 50 or the scanning system 40 by requesting the compressedimage tiles at a specified pyramid level on an as-needed basis and byperforming client side caching of tiles in anticipation of userrequests.

The digital visualization and analysis module 520 additionally operatesto facilitate whole slide quantitative analysis of the digital slideimages, whether the image is a quadrant style image or a fused styleimage. In one embodiment, the digital visualization and analysis module520 can facilitate a quantitative analysis of a particular region ofinterest instead of the entire digital slide image. Analysis results canbe stored in a data storage area such as data storage areas 445, 455, or435 for use with data management and reporting.

FIG. 5A illustrates a block diagram of a preferred embodiment of anoptical microscopy system 10 according to the present invention isshown. The heart of the system 10 is a microscope slide scanner 11 thatserves to scan and digitize a specimen or sample 12. The sample 12 canbe anything that may be interrogated by optical microscopy. Forinstance, the sample 12 may be a microscope slide or other sample typethat may be interrogated by optical microscopy. A microscope slide isfrequently used as a viewing substrate for specimens that includetissues and cells, chromosomes, DNA, protein, blood, bone marrow, urine,bacteria, beads, biopsy materials, or any other type of biologicalmaterial or substance that is either dead or alive, stained orunstained, labeled or unlabeled. The sample 12 may also be an array ofany type of DNA or DNA-related material such as cDNA or RNA or proteinthat is deposited on any type of slide or other substrate, including anyand all samples commonly known as a microarrays. The sample 12 may be amicrotiter plate, for example a 96-well plate. Other examples of thesample 12 include integrated circuit boards, electrophoresis records,petri dishes, film, semiconductor materials, forensic materials, ormachined parts.

The scanner 11 includes a motorized stage 14, a microscope objectivelens 16, a line scan camera 18, and a data processor 20. The sample 12is positioned on the motorized stage 14 for scanning. The motorizedstage 14 is connected to a stage controller 22 which is connected inturn to the data processor 20. The data processor 20 determines theposition of the sample 12 on the motorized stage 14 via the stagecontroller 22. In the presently preferred embodiment, the motorizedstage 14 moves the sample 12 in at least the two axes (x/y) that are inthe plane of the sample 12. Fine movements of the sample 12 along theoptical z-axis may also be necessary for certain applications of thescanner 11, for example, for focus control. Z-axis movement ispreferably accomplished with a piezo positioner 24, such as the PIFOCfrom Polytec PI or the MIPOS 3 from Piezosystem Jena. The piezopositioner 24 is attached directly to the microscope objective 16 and isconnected to and directed by the data processor 20 via a piezocontroller 26. A means of providing a coarse focus adjustment may alsobe needed and can be provided by z-axis movement as part of themotorized stage 14 or a manual rack-and-pinion coarse focus adjustment(not shown).

In the presently preferred embodiment, the motorized stage 14 includes ahigh precision positioning table with ball bearing linear ways toprovide smooth motion and excellent straight line and flatness accuracy.For example, the motorized stage 14 could include two Daedal model106004 tables stacked one on top of the other. Other types of motorizedstages 14 are also suitable for the scanner 11, including stacked singleaxis stages based on ways other than ball bearings, single- ormultiple-axis positioning stages that are open in the center and areparticularly suitable for trans-illumination from below the sample, orlarger stages that can support a plurality of samples. In the presentlypreferred embodiment, motorized stage 14 includes two stackedsingle-axis positioning tables, each coupled to two millimeterlead-screws and Nema-23 stepping motors. At the maximum lead screw speedof twenty-five revolutions per second, the maximum speed of the sample12 on the motorized stage 14 is fifty millimeters per second. Selectionof a lead screw with larger diameter, for example five millimeters, canincrease the maximum speed to more than 100 millimeters per second. Themotorized stage 14 can be equipped with mechanical or optical positionencoders which has the disadvantage of adding significant expense to thesystem. Consequently, the presently preferred embodiment does notinclude position encoders. However, if one were to use servo motors inplace of stepping motors, then one would have to use position feedbackfor proper control.

Position commands from the data processor 20 are converted to motorcurrent or voltage commands in the stage controller 22. In the presentlypreferred embodiment, the stage controller 22 includes a 2-axisservo/stepper motor controller (Compumotor 6K2) and two 4-ampmicrostepping drives (Compumotor OEMZL4). Microstepping provides a meansfor commanding the stepper motor in much smaller increments than therelatively large single 1.8 degree motor step. For example, at amicrostep of 100, the sample 12 can be commanded to move at steps assmall as 0.1 micrometer. A microstep of 25,000 is used in the presentlypreferred embodiment of this invention. Smaller step sizes are alsopossible. It should be obvious that the optimum selection of themotorized stage 14 and the stage controller 22 depends on many factors,including the nature of the sample 12, the desired time for sampledigitization, and the desired resolution of the resulting digital imageof the sample 12.

The microscope objective lens 16 can be any microscope objective lenscommonly available. One of ordinary skill in the art will realize thatthe choice of which objective lens to use will depend on the particularcircumstances. In the preferred embodiment of the present invention, themicroscope objective lens 16 is of the infinity-corrected type.

The sample 12 is illuminated by an illumination system 28 that includesa light source 30 and illumination optics 32. The light source 30 in thepresently preferred embodiment includes a variable intensity halogenlight source with a concave reflective mirror to maximize light outputand a KG-1 filter to suppress heat. However, the light source 30 couldalso be any other type of arc-lamp, laser, or other source of light. Theillumination optics 32 in the presently preferred embodiment include astandard Köhler illumination system with two conjugate planes that areorthogonal to the optical axis. The illumination optics 32 arerepresentative of the bright-field illumination optics that can be foundon most commercially available compound microscopes sold by companiessuch as Carl Zeiss, Nikon, Olympus, or Leica. One set of conjugateplanes includes (i) a field iris aperture illuminated by the lightsource 30, (ii) the object plane that is defined by the focal plane ofthe sample 12, and (iii) the plane containing the light-responsiveelements of the line scan camera 18. A second conjugate plane includes(i) the filament of the bulb that is part of the light source 30, (ii)the aperture of a condenser iris that sits immediately before thecondenser optics that are part of the illumination optics 32, and (iii)the back focal plane of the microscope objective lens 16. In thepresently preferred embodiment, the sample 12 is illuminated and imagedin transmission mode, with the line scan camera 18 sensing opticalenergy that is transmitted by the sample 12, or conversely, opticalenergy that is absorbed by the sample 12.

The scanner 11 of the present invention is equally suitable fordetecting optical energy that is reflected from the sample 12, in whichcase the light source 30, the illumination optics 32, and the microscopeobjective lens 16 must be selected based on compatibility withreflection imaging. One possible embodiment may therefore beillumination through a fiber optic bundle that is positioned above thesample 12. Other possibilities include excitation that is spectrallyconditioned by a monochromator. If the microscope objective lens 16 isselected to be compatible with phase-contrast microscopy, then theincorporation of at least one phase stop in the condenser optics thatare part of the illumination optics 32 will enable the scanner 11 to beused for phase contrast microscopy. To one of ordinary skill in the art,the modifications required for other types of microscopy such asdifferential interference contrast and confocal microscopy should bereadily apparent. Overall, the scanner 11 is suitable, with appropriatebut well-known modifications, for the interrogation of microscopicsamples in any known mode of optical microscopy.

Between the microscope objective lens 16 and the line scan camera 18 aresituated the line scan camera focusing optics 34 that focus the opticalsignal captured by the microscope objective lens 16 onto thelight-responsive elements of the line scan camera 18. In a moderninfinity-corrected microscope the focusing optics between the microscopeobjective lens and the eyepiece optics, or between the microscopeobjective lens and an external imaging port, consist of an opticalelement known as a tube lens that is part of a microscope's observationtube. Many times the tube lens consists of multiple optical elements toprevent the introduction of coma or astigmatism. One of the motivationsfor the relatively recent change from traditional finite tube lengthoptics to infinity corrected optics was to increase the physical spacein which the optical energy from the sample 12 is parallel, meaning thatthe focal point of this optical energy is at infinity. In this case,accessory elements like dichroic mirrors or filters can be inserted intothe infinity space without changing the optical path magnification orintroducing undesirable optical artifacts.

Infinity-corrected microscope objective lenses are typically inscribedwith an infinity mark. The magnification of an infinity correctedmicroscope objective lens is given by the quotient of the focal lengthof the tube lens divided by the focal length of the objective lens. Forexample, a tube lens with a focal length of 180 millimeters will resultin 20× magnification if an objective lens with 9 millimeter focal lengthis used. One of the reasons that the objective lenses manufactured bydifferent microscope manufacturers are not compatible is because of alack of standardization in the tube lens focal length. For example, a20× objective lens from Olympus, a company that uses a 180 millimetertube lens focal length, will not provide a 20× magnification on a Nikonmicroscope that is based on a different tube length focal length of 200millimeters. Instead, the effective magnification of such an Olympusobjective lens engraved with 20× and having a 9 millimeter focal lengthwill be 22.2×, obtained by dividing the 200 millimeter tube lens focallength by the 9 millimeter focal length of the objective lens. Changingthe tube lens on a conventional microscope is virtually impossiblewithout disassembling the microscope. The tube lens is part of acritical fixed element of the microscope. Another contributing factor tothe incompatibility between the objective lenses and microscopesmanufactured by different manufacturers is the design of the eyepieceoptics, the binoculars through which the specimen is observed. Whilemost of the optical corrections have been designed into the microscopeobjective lens, most microscope users remain convinced that there issome benefit in matching one manufacturers' binocular optics with thatsame manufacturers' microscope objective lenses to achieve the bestvisual image.

The line scan camera focusing optics 34 include a tube lens opticmounted inside of a mechanical tube. Since the scanner 11, in itspreferred embodiment, lacks binoculars or eyepieces for traditionalvisual observation, the problem suffered by conventional microscopes ofpotential incompatibility between objective lenses and binoculars isimmediately eliminated. One of ordinary skill will similarly realizethat the problem of achieving parfocality between the eyepieces of themicroscope and a digital image on a display monitor is also eliminatedby virtue of not having any eyepieces. Since the scanner 11 alsoovercomes the field of view limitation of a traditional microscope byproviding a field of view that is practically limited only by thephysical boundaries of the sample 12, the importance of magnification inan all-digital imaging microscope such as provided by the presentscanner 11 is limited. Once a portion of the sample 12 has beendigitized, it is straightforward to apply electronic magnification,sometimes known as electric zoom, to an image of the sample 12 in orderto increase its magnification. Increasing the magnification of an imageelectronically has the effect of increasing the size of that image onthe monitor that is used to display the image. If too much electroniczoom is applied, then the display monitor will be able to show onlyportions of the magnified image. It is not possible, however, to useelectronic magnification to display information that was not present inthe original optical signal that was digitized in the first place. Sinceone of the objectives of the scanner 11 is to provide high qualitydigital images, in lieu of visual observation through the eyepieces of amicroscope, it is important that the content of the images acquired bythe scanner 11 include as much image detail as possible. The termresolution is typically used to describe such image detail and the termdiffraction-limited is used to describe the wavelength-limited maximumspatial detail available in an optical signal. The scanner 11 providesdiffraction-limited digital imaging by selection of a tube lens focallength that is matched according to the well know Nyquist samplingcriteria to both the size of an individual pixel element in alight-sensing camera such as the line scan camera 18 and to thenumerical aperture of the microscope objective lens 16. It is well knownthat numerical aperture, not magnification, is the resolution-limitingattribute of a microscope objective lens 16.

An example will help to illustrate the optimum selection of a tube lensfocal length that is part of the line scan camera focusing optics 34.Consider again the 20× microscope objective lens 16 with 9 millimeterfocal length discussed previously and assume that this objective lenshas a numerical aperture of 0.50. Assuming no appreciable degradationfrom the condenser, the diffraction-limited resolving power of thisobjective lens at a wavelength of 500 nanometers is approximately 0.6micrometers, obtained using the well-known Abbe relationship. Assumefurther that the line scan camera 18, which in its preferred embodimenthas a plurality of 14 micrometer square pixels, is used to detect aportion of the sample 12. In accordance with sampling theory, it isnecessary that at least two sensor pixels subtend the smallestresolvable spatial feature. In this case, the tube lens must be selectedto achieve a magnification of 46.7, obtained by dividing 28 micrometers,which corresponds to two 14 micrometer pixels, by 0.6 micrometers, thesmallest resolvable feature dimension. The optimum tube lens optic focallength is therefore about 420 millimeters, obtained by multiplying 46.7by 9. The line scan focusing optics 34 with a tube lens optic having afocal length of 420 millimeters will therefore be capable of acquiringimages with the best possible spatial resolution, similar to what wouldbe observed by viewing a specimen under a microscope using the same 20×objective lens. To reiterate, the scanner 11 utilizes a traditional 20×microscope objective lens 16 in a higher magnification opticalconfiguration, in this example about 47×, in order to acquirediffraction-limited digital images. If a traditional 20× magnificationobjective lens 16 with a higher numerical aperture were used, say 0.75,the required tube lens optic magnification for diffraction-limitedimaging would be about 615 millimeters, corresponding to an overalloptical magnification of 68×. Similarly, if the numerical aperture ofthe 20× objective lens were only 0.3, the optimum tube lens opticmagnification would only be about 28×, which corresponds to a tube lensoptic focal length of approximately 252 millimeters. The line scancamera focusing optics 34 are modular elements of the scanner 11 and canbe interchanged as necessary for optimum digital imaging. The advantageof diffraction-limited digital imaging is particularly significant forapplications, for example bright field microscopy, in which thereduction in signal brightness that accompanies increases inmagnification is readily compensated by increasing the intensity of anappropriately designed illumination system 28.

In principle, it is possible to attach external magnification-increasingoptics to a conventional microscope-based digital imaging system toeffectively increase the tube lens magnification so as to achievediffraction-limited imaging as has just been described for the presentscanner 11; however, the resulting decrease in the field of view isoften unacceptable, making this approach impractical. Furthermore, manyusers of microscopes typically do not understand enough about thedetails of diffraction-limited imaging to effectively employ thesetechniques on their own. In practice, digital cameras are attached tomicroscope ports with magnification-decreasing optical couplers toattempt to increase the size of the field of view to something moresimilar to what can be seen through the eyepiece. The standard practiceof adding de-magnifying optics is a step in the wrong direction if thegoal is to obtain diffraction-limited digital images.

In a conventional microscope, different power objectives lenses aretypically used to view the specimen at different resolutions andmagnifications. Standard microscopes have a nosepiece that holds fiveobjectives lenses. In an all-digital imaging system such as the presentscanner 11 there is a need for only one microscope objective lens 16with a numerical aperture corresponding to the highest spatialresolution desirable. The presently preferred embodiment of the scanner11 provides for only one microscope objective lens 16. Once adiffraction-limited digital image has been captured at this resolution,it is straightforward using standard digital image processingtechniques, to present imagery information at any desirable reducedresolutions and magnifications.

The presently preferred embodiment of the scanner 11 is based on a DalsaSPARK line scan camera 18 with 1024 pixels (picture elements) arrangedin a linear array, with each pixel having a dimension of 14 by 14micrometers. Any other type of linear array, whether packaged as part ofa camera or custom-integrated into an imaging electronic module, canalso be used. The linear array in the presently preferred embodimenteffectively provides eight bits of quantization, but other arraysproviding higher or lower level of quantization may also be used.Alternate arrays based on 3-channel red-green-blue (RGB) colorinformation or time delay integration (TDI), may also be used. TDIarrays provide a substantially better signal-to-noise ratio (SNR) in theoutput signal by summing intensity data from previously imaged regionsof a specimen, yielding an increase in the SNR that is in proportion tothe square-root of the number of integration stages. TDI arrays cancomprise multiple stages of linear arrays. TDI arrays are available with24, 32, 48, 64, 96, or even more stages. The scanner 11 also supportslinear arrays that are manufactured in a variety of formats includingsome with 512 pixels, some with 1024 pixels, and others having as manyas 4096 pixels. Appropriate, but well known, modifications to theillumination system 28 and the line scan camera focusing optics 34 maybe required to accommodate larger arrays. Linear arrays with a varietyof pixel sizes can also be used in scanner 11. The salient requirementfor the selection of any type of line scan camera 18 is that the sample12 can be in motion with respect to the line scan camera 18 during thedigitization of the sample 12 in order to obtain high quality images,overcoming the static requirements of the conventional imaging tilingapproaches known in the prior art.

The output signal of the line scan camera 18 is connected to the dataprocessor 20. The data processor 20 in the presently preferredembodiment includes a central processing unit with ancillaryelectronics, for example a motherboard, to support at least one signaldigitizing electronics board such as an imaging board or a framegrabber. In the presently preferred embodiment, the imaging board is anEPIX PIXCID24 PCI bus imaging board, however, there are many other typesof imaging boards or frame grabbers from a variety of manufacturerswhich could be used in place of the EPIX board. An alternate embodimentcould be a line scan camera that uses an interface such as IEEE 1394,also known as Firewire, to bypass the imaging board altogether and storedata directly on a data storage 38, such as a hard disk.

The data processor 20 is also connected to a memory 36, such as randomaccess memory (RAM), for the short-term storage of data, and to the datastorage 38, such as a hard drive, for long-term data storage. Further,the data processor 20 is connected to a communications port 40 that isconnected to a network 42 such as a local area network (LAN), a widearea network (WAN), a metropolitan area network (MAN), an intranet, anextranet, or the global Internet. The memory 36 and the data storage 38are also connected to each other. The data processor 20 is also capableof executing computer programs, in the form of software, to controlcritical elements of the scanner 11 such as the line scan camera 18 andthe stage controller 22, or for a variety of image-processing functions,image-analysis functions, or networking. The data processor 20 can bebased on any operating system, including operating systems such asWindows, Linux, OS/2, Mac OS, and Unix. In the presently preferredembodiment, the data processor 20 operates based on the Windows NToperating system.

The data processor 20, memory 36, data storage 38, and communicationport 40 are each elements that can be found in a conventional computer.One example would be a personal computer such as a Dell Dimension XPST500 that features a Pentium III 500 MHz processor and up to 756megabytes (MB) of RAM. In the presently preferred embodiment, thecomputer, elements which include the data processor 20, memory 36, datastorage 38, and communications port 40 are all internal to the scanner11, so that the only connection of the scanner 11 to the other elementsof the system 10 is the communication port 40. In an alternateembodiment of the scanner 11, the computer elements would be external tothe scanner 11 with a corresponding connection between the computerelements and the scanner 11.

The scanner 11, in the presently preferred embodiment of the invention,integrates optical microscopy, digital imaging, motorized samplepositioning, computing, and network-based communications into asingle-enclosure unit. The major advantage of packaging the scanner 11as a single-enclosure unit with the communications port 40 as theprimary means of data input and output are reduced complexity andincreased reliability. The various elements of the scanner 11 areoptimized to work together, in sharp contrast to traditionalmicroscope-based imaging systems in which the microscope, light source,motorized stage, camera, and computer are typically provided bydifferent vendors and require substantial integration and maintenance.

The communication port 40 provides a means for rapid communications withthe other elements of the system 10, including the network 42. Thepresently preferred communications protocol for the communications port40 is a carrier-sense multiple-access collision detection protocol suchas Ethernet, together with the TCP/IP protocol for transmission controland internetworking. The scanner 11 is intended to work with any type oftransmission media, including broadband, baseband, coaxial cable,twisted pair, fiber optics, DSL or wireless.

In the presently preferred embodiment, control of the scanner 11 andreview of the imagery data captured by the scanner 11 are performed on acomputer 44 that is connected to the network 42. The computer 44, in itspresently preferred embodiment, is connected to a display monitor 46 toprovide imagery information to an operator. A plurality of computers 44may be connected to the network 42. In the presently preferredembodiment, the computer 44 communicates with the scanner 11 using anetwork browser such as Internet Explorer from Microsoft or NetscapeCommunicator from AOL. Images are stored on the scanner 11 in a commoncompressed format such a JPEG which is an image format that iscompatible with standard image-decompression methods that are alreadybuilt into most commercial browsers. Other standard or non-standard,lossy or lossless, image compression formats will also work. In thepresently preferred embodiment, the scanner 11 is a webserver providingan operator interface that is based on webpages that are sent from thescanner 11 to the computer 44. For dynamic review of imagery data, thecurrently preferred embodiment of the scanner 11 is based on playingback, for review on the display monitor 46 that is connected to thecomputer 44, multiple frames of imagery data using standardmultiple-frame browser compatible software packages such as Media-Playerfrom Microsoft, Quicktime from Apple Computer, or RealPlayer from RealNetworks. In the presently preferred embodiment, the browser on thecomputer 44 uses the hypertext transmission protocol (http) togetherwith TCP for transmission control.

There are, and will be in the future, many different means and protocolsby which the scanner 11 could communicate with the computer 44, or aplurality of computers. While the presently preferred embodiment isbased on standard means and protocols, the approach of developing one ormultiple customized software modules known as applets is equallyfeasible and may be desirable for selected future applications of thescanner 11. Further, there are no constraints that computer 44 be of anyspecific type such as a personal computer (PC) or be manufactured by anyspecific company such as Dell. One of the advantages of a standardizedcommunications port 40 is that any type of computer 44 operating commonnetwork browser software can communicate with the scanner 11.

If one so desires, it is possible, with some modifications to thescanner 11, to obtain spectrally resolved images. Spectrally resolvedimages are images in which spectral information is measured at everyimage pixel. Spectrally resolved images could be obtained by replacingthe line scan camera 18 of the scanner 11 with an optical slit and animaging spectrograph. The imaging spectrograph uses a two-dimensionalCCD detector to capture wavelength-specific intensity data for a columnof image pixels by using a prism or grating to disperse the opticalsignal that is focused on the optical slit along each of the rows of thedetector.

FIG. 5B illustrates a block diagram of a second embodiment of an opticalmicroscopy system 10 according to the present invention is shown. Inthis system 10, the scanner 11 is more complex and expensive than thecurrently preferred embodiment shown in FIG. 1. The additionalattributes of the scanner 11 that are shown do not all have to bepresent for any alternate embodiment to function correctly. FIG. 2 isintended to provide a reasonable example of additional features andcapabilities that could be incorporated into the scanner 11.

The alternate embodiment of FIG. 2 provides for a much greater level ofautomation than the presently preferred embodiment of FIG. 1. A morecomplete level of automation of the illumination system 28 is achievedby connections between the data processor 20 and both the light source30 and the illumination optics 32 of the illumination system 28. Theconnection to the light source 30 may control the voltage, or current,in an open or closed loop fashion, in order to control the intensity ofthe light source 30. Recall that the light source 30 is a halogen bulbin the presently preferred embodiment. The connection between the dataprocessor 20 and the illumination optics 32 could provide closed loopcontrol of the field iris aperture and the condenser iris to provide ameans for ensuring that optimum Köhler illumination is maintained.

Use of the scanner 11 for fluorescence imaging requires easilyrecognized modifications to the light source 30, the illumination optics32, and the microscope objective lens 16. The second embodiment of FIG.2 also provides for a fluorescence filter cube 50 that includes anexcitation filter, a dichroic filter, and a barrier filter. Thefluorescence filter cube 50 is positioned in the infinity corrected beampath that exists between the microscope objective lens 16 and line scancamera focusing optics 34. One embodiment for fluorescence imaging couldinclude the addition of a filter wheel or tunable filter into theillumination optics 32 to provide appropriate spectral excitation forthe variety of fluorescent dyes or nano-crystals available on themarket.

The addition of at least one beam splitter 52 into the imaging pathallows the optical signal to be split into at least two paths. Theprimary path is via the line scan camera focusing optics 34, asdiscussed previously, to enable diffraction-limited imaging by the linescan camera 18. A second path is provided via an area scan camerafocusing optics 54 for imaging by an area scan camera 56. It should bereadily apparent that proper selection of these two focusing optics canensure diffraction-limited imaging by the two camera sensors havingdifferent pixel sizes. The area scan camera 56 can be one of many typesthat are currently available, including a simple color video camera, ahigh performance, cooled, CCD camera, or a variable integration-timefast frame camera. The area scan camera 56 provides a traditionalimaging system configuration for the scanner 11. The area scan camera 56is connected to the data processor 20. If two cameras are used, forexample the line scan camera 18 and the area scan camera 56, both cameratypes could be connected to the data processor using either a singledual-purpose imaging board, two different imaging boards, or theIEEE1394 Firewire interface, in which case one or both imaging boardsmay not be needed. Other related methods of interfacing imaging sensorsto the data processor 20 are also available.

While the primary interface of the scanner 11 to the computer 44 is viathe network 42, there may be instances, for example a failure of thenetwork 42, where it is beneficial to be able to connect the scanner 11directly to a local output device such as a display monitor 58 and toalso provide local input devices such as a keyboard and mouse 60 thatare connected directly into the data processor 20 of the scanner 11. Inthis instance, the appropriate driver software and hardware would haveto be provided as well.

The second embodiment shown in FIG. 2 also provides for a much greaterlevel of automated imaging performance. Enhanced automation of theimaging of the scanner 11 can be achieved by closing the focus controlloop comprising the piezo positioner 24, the piezo controller 26, andthe data processor 20 using well-known methods of autofocus. The secondembodiment also provides for a motorized nose-piece 62 to accommodateseveral objectives lenses. The motorized nose-piece 62 is connected toand directed by the data processor 20 through a nose-piece controller64.

There are other features and capabilities of the scanner 11 which couldbe incorporated. For example, the process of scanning the sample 12 withrespect to the microscope objective lens 16 that is substantiallystationary in the x/y plane of the sample 12 could be modified tocomprise scanning of the microscope objective lens 16 with respect to astationary sample 12. Scanning the sample 12, or scanning the microscopeobjective lens 16, or scanning both the sample 12 and the microscopeobjective lens 16 simultaneously, are possible embodiments of thescanner 11 which can provide the same large contiguous digital image ofthe sample 12 as discussed previously.

The scanner 11 also provides a general purpose platform for automatingmany types of microscope-based analyses. The illumination system 28could be modified from a traditional halogen lamp or arc-lamp to alaser-based illumination system to permit scanning of the sample 12 withlaser excitation. Modifications, including the incorporation of aphotomultiplier tube or other non-imaging detector, in addition to or inlieu of the line scan camera 18 or the area scan camera 56, could beused to provide a means of detecting the optical signal resulting fromthe interaction of the laser energy with the sample 12.

FIG. 5C is a block diagram of a third embodiment of an opticalmicroscopy system 10 according to the present invention. In this system10, the scanner 11 is optimized for scanning fluorescent microscopesamples. The additional attributes of the scanner 11 in this embodimentincluding the various software and hardware elements do not all have tobe present for operation of the fluorescence scanner to functioncorrectly. FIG. 3 illustrates a reasonable example of additionalfeatures and capabilities that could be incorporated into the scanner 11for scanning fluorescent microscope samples.

FIG. 13 is a block diagram illustrating an example computer system 550that may be used in connection with various embodiments describedherein. For example, the computer system 550 may be used in conjunctionwith the digital pathology system and the computer and display monitorsused in conjunction with the viewing software described herein. However,other computer systems and/or architectures may be used, as will beclear to those skilled in the art.

The computer system 550 preferably includes one or more processors, suchas processor 552. Additional processors may be provided, such as anauxiliary processor to manage input/output, an auxiliary processor toperform floating point mathematical operations, a special-purposemicroprocessor having an architecture suitable for fast execution ofsignal processing algorithms (e.g., digital signal processor), a slaveprocessor subordinate to the main processing system (e.g., back-endprocessor), an additional microprocessor or controller for dual ormultiple processor systems, or a coprocessor. Such auxiliary processorsmay be discrete processors or may be integrated with the processor 552.

The processor 552 is preferably connected to a communication bus 554.The communication bus 554 may include a data channel for facilitatinginformation transfer between storage and other peripheral components ofthe computer system 550. The communication bus 554 further may provide aset of signals used for communication with the processor 552, includinga data bus, address bus, and control bus (not shown). The communicationbus 554 may comprise any standard or non-standard bus architecture suchas, for example, bus architectures compliant with industry standardarchitecture (“ISA”), extended industry standard architecture (“EISA”),Micro Channel Architecture (“MCA”), peripheral component interconnect(“PCI”) local bus, or standards promulgated by the Institute ofElectrical and Electronics Engineers (“IEEE”) including IEEE 488general-purpose interface bus (“GPIB”), IEEE 696/S-100, and the like.

Computer system 550 preferably includes a main memory 556 and may alsoinclude a secondary memory 558. The main memory 556 provides storage ofinstructions and data for programs executing on the processor 552. Themain memory 556 is typically semiconductor-based memory such as dynamicrandom access memory (“DRAM”) and/or static random access memory(“SRAM”). Other semiconductor-based memory types include, for example,synchronous dynamic random access memory (“SDRAM”), Rambus dynamicrandom access memory (“RDRAM”), ferroelectric random access memory(“FRAM”), and the like, including read only memory (“ROM”).

The secondary memory 558 may optionally include a hard disk drive 560and/or a removable storage drive 562, for example a floppy disk drive, amagnetic tape drive, a compact disc (“CD”) drive, a digital versatiledisc (“DVD”) drive, etc. The removable storage drive 562 reads fromand/or writes to a removable storage medium 564 in a well-known manner.Removable storage medium 564 may be, for example, a floppy disk,magnetic tape, CD, DVD, etc.

The removable storage medium 564 is preferably a computer readablemedium having stored thereon computer executable code (i.e., software)and/or data. The computer software or data stored on the removablestorage medium 564 is read into the computer system 550 as electricalcommunication signals 578.

In alternative embodiments, secondary memory 558 may include othersimilar means for allowing computer programs or other data orinstructions to be loaded into the computer system 550. Such means mayinclude, for example, an external storage medium 572 and an interface570. Examples of external storage medium 572 may include an externalhard disk drive or an external optical drive, or and externalmagneto-optical drive.

Other examples of secondary memory 558 may include semiconductor-basedmemory such as programmable read-only memory (“PROM”), erasableprogrammable read-only memory (“EPROM”), electrically erasable read-onlymemory (“EEPROM”), or flash memory (block oriented memory similar toEEPROM). Also included are any other removable storage units 572 andinterfaces 570, which allow software and data to be transferred from theremovable storage unit 572 to the computer system 550.

Computer system 550 may also include a communication interface 574. Thecommunication interface 574 allows software and data to be transferredbetween computer system 550 and external devices (e.g. printers),networks, or information sources. For example, computer software orexecutable code may be transferred to computer system 550 from a networkserver via communication interface 574. Examples of communicationinterface 574 include a modem, a network interface card (“NIC”), acommunications port, a PCMCIA slot and card, an infrared interface, andan IEEE 1394 fire-wire, just to name a few.

Communication interface 574 preferably implements industry promulgatedprotocol standards, such as Ethernet IEEE 802 standards, Fiber Channel,digital subscriber line (“DSL”), asynchronous digital subscriber line(“ADSL”), frame relay, asynchronous transfer mode (“ATM”), integrateddigital services network (“ISDN”), personal communications services(“PCS”), transmission control protocol/Internet protocol (“TCP/IP”),serial line Internet protocol/point to point protocol (“SLIP/PPP”), andso on, but may also implement customized or non-standard interfaceprotocols as well.

Software and data transferred via communication interface 574 aregenerally in the form of electrical communication signals 578. Thesesignals 578 are preferably provided to communication interface 574 via acommunication channel 576. Communication channel 576 carries signals 578and can be implemented using a variety of wired or wirelesscommunication means including wire or cable, fiber optics, conventionalphone line, cellular phone link, wireless data communication link, radiofrequency (RF) link, or infrared link, just to name a few.

Computer executable code (i.e., computer programs or software) is storedin the main memory 556 and/or the secondary memory 558. Computerprograms can also be received via communication interface 574 and storedin the main memory 556 and/or the secondary memory 558. Such computerprograms, when executed, enable the computer system 550 to perform thevarious functions of the present invention as previously described.

In this description, the term “computer readable medium” is used torefer to any media used to provide computer executable code (e.g.,software and computer programs) to the computer system 550. Examples ofthese media include main memory 556, secondary memory 558 (includinghard disk drive 560, removable storage medium 564, and external storagemedium 572), and any peripheral device communicatively coupled withcommunication interface 574 (including a network information server orother network device). These computer readable mediums are means forproviding executable code, programming instructions, and software to thecomputer system 550.

In an embodiment that is implemented using software, the software may bestored on a computer readable medium and loaded into computer system 550by way of removable storage drive 562, interface 570, or communicationinterface 574. In such an embodiment, the software is loaded into thecomputer system 550 in the form of electrical communication signals 578.The software, when executed by the processor 552, preferably causes theprocessor 552 to perform the inventive features and functions previouslydescribed herein.

Various embodiments may also be implemented primarily in hardware using,for example, components such as application specific integrated circuits(“ASICs”), or field programmable gate arrays (“FPGAs”). Implementationof a hardware state machine capable of performing the functionsdescribed herein will also be apparent to those skilled in the relevantart. Various embodiments may also be implemented using a combination ofboth hardware and software.

Furthermore, those of skill in the art will appreciate that the variousillustrative logical blocks, modules, circuits, and method stepsdescribed in connection with the above described figures and theembodiments disclosed herein can often be implemented as electronichardware, computer software, or combinations of both. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, circuits, and steps have beendescribed above generally in terms of their functionality. Whether suchfunctionality is implemented as hardware or software depends upon theparticular application and design constraints imposed on the overallsystem. Skilled persons can implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the invention. In addition, the grouping of functions within amodule, block, circuit or step is for ease of description. Specificfunctions or steps can be moved from one module, block or circuit toanother without departing from the invention.

Moreover, the various illustrative logical blocks, modules, and methodsdescribed in connection with the embodiments disclosed herein can beimplemented or performed with a general purpose processor, a digitalsignal processor (“DSP”), an ASIC, FPGA or other programmable logicdevice, discrete gate or transistor logic, discrete hardware components,or any combination thereof designed to perform the functions describedherein. A general-purpose processor can be a microprocessor, but in thealternative, the processor can be any processor, controller,microcontroller, or state machine. A processor can also be implementedas a combination of computing devices, for example, a combination of aDSP and a microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

Additionally, the steps of a method or algorithm described in connectionwith the embodiments disclosed herein can be embodied directly inhardware, in a software module executed by a processor, or in acombination of the two. A software module can reside in RAM memory,flash memory, ROM memory, EPROM memory, EEPROM memory, registers, harddisk, a removable disk, a CD-ROM, or any other form of storage mediumincluding a network storage medium. An exemplary storage medium can becoupled to the processor such the processor can read information from,and write information to, the storage medium. In the alternative, thestorage medium can be integral to the processor. The processor and thestorage medium can also reside in an ASIC.

The above description of the disclosed embodiments is provided to enableany person skilled in the art to make or use the invention. Variousmodifications to these embodiments will be readily apparent to thoseskilled in the art, and the generic principles described herein can beapplied to other embodiments without departing from the spirit or scopeof the invention. Thus, it is to be understood that the description anddrawings presented herein represent a presently preferred embodiment ofthe invention and are therefore representative of the subject matterwhich is broadly contemplated by the present invention. It is furtherunderstood that the scope of the present invention fully encompassesother embodiments that may become obvious to those skilled in the artand that the scope of the present invention is accordingly not limited.

What is claimed is:
 1. A method for assessing a quality of a digitalimage, the method comprising using at least one hardware processor to:perform a micro-analysis comprising dividing a digital image into aplurality of blocks, for two or more of the plurality of blocks,determining a score based on a spatial frequency of the block, andgenerating a score map for the digital image based on the score for eachof the two or more blocks; and perform a macro-analysis comprisingdetecting artifacts in the digital image, computing a degradation scorebased on detected artifacts, and computing a whole-slide-quality scorebased on the score map and the degradation score.
 2. The method of claim1, wherein the micro-analysis further comprises: for each of theplurality of blocks, generating one or more metrics for the block,qualifying the block if the one or more metrics satisfy one or morepredetermined criteria, and not qualifying the block if the one or moremetrics do not satisfy the one or more predetermined criteria, whereinthe two or more blocks consist of the qualified ones of the plurality ofblocks.
 3. The method of claim 2, wherein the one or more metricscomprise one or more of a measure of white space, a measure of light, ameasure of edge activity, and a measure of contrast.
 4. The method ofclaim 2, wherein the macro-analysis further comprises determining aratio of a number of the qualified blocks to a total number of theplurality of blocks.
 5. The method of claim 1, wherein determining ascore based on a spatial frequency of the block comprises: performing atransform on at least a portion of the block; and generating the scorebased on the transform.
 6. The method of claim 5, wherein performing atransform on at least a portion of the block comprises performing aplurality of partial transforms, wherein each of the plurality ofpartial transforms is performed on at least a portion of the block, andwherein the score is based on the plurality of partial transforms. 7.The method of claim 6, wherein the score is based on a number of theplurality of partial transforms exceeding a predetermined threshold. 8.The method of claim 7, wherein the score is based on a ratio of thenumber of the plurality of partial transforms exceeding thepredetermined threshold to a total number of the plurality of partialtransforms.
 9. The method of claim 6, further comprising weighting theplurality of partial transforms to emphasize a contribution of one ormore frequency ranges, wherein the score is based on the weightedplurality of partial transforms.
 10. The method of claim 1, whereincomputing a whole-slide-quality score based on the score map and thedegradation score comprises: computing an average score from the scoremap; and subtracting the degradation score from the average score. 11.The method of claim 1, wherein the macro-analysis further comprises, ifthe whole-slide-quality score does not satisfy a predeterminedthreshold, providing an indication that the digital image hasunsatisfactory image quality.
 12. The method of claim 1, furthercomprising generating a heat map for the digital image based on thescore map, wherein the heat map comprises the digital image with a coloroverlaid on each of the plurality of blocks, and wherein, for each ofthe plurality of blocks, the overlaid color represents an image qualityof the block.
 13. The method of claim 12, wherein generating the heatmap comprises, for each of the plurality of blocks, blending a grayscaleimage corresponding to the block with a color representing an imagequality of the block.
 14. The method of claim 1, wherein the score mapcomprises a plurality of scores for the plurality of blocks, and whereindetecting artifacts in the digital image comprises searching for one ormore patterns in the plurality of scores.
 15. The method of claim 14,wherein the one or more patterns comprise a discontinuity in ahorizontal or vertical dimension of the digital image.
 16. Anon-transitory computer-readable medium having instructions storedtherein, wherein the instructions, when executed by a processor, causethe processor to: perform a micro-analysis comprising dividing a digitalimage into a plurality of blocks, for two or more of the plurality ofblocks, determining a score based on a spatial frequency of the block,and generating a score map for the digital image based on the score foreach of the two or more blocks; and perform a macro-analysis comprisingdetecting artifacts in the digital image, computing a degradation scorebased on detected artifacts, and computing a whole-slide-quality scorebased on the score map and the degradation score.
 17. The non-transitorycomputer-readable medium of claim 16, wherein computing awhole-slide-quality score based on the score map and the degradationscore comprises: computing an average score from the score map; andsubtracting the degradation score from the average score.
 18. Thenon-transitory computer-readable medium of claim 16, wherein theinstructions further cause the processor to generate a heat map for thedigital image based on the score map, wherein the heat map comprises thedigital image with a color overlaid on each of the plurality of blocks,and wherein, for each of the plurality of blocks, the overlaid colorrepresents an image quality of the block.
 19. The non-transitorycomputer-readable medium of claim 16, wherein the score map comprises aplurality of scores for the plurality of blocks, and wherein detectingartifacts in the digital image comprises searching for one or morepatterns in the plurality of scores.
 20. A system for assessing aquality of a digital slide image, the system comprising: at least onehardware processor; and one or more modules that are configured to, whenexecuted by the at least one hardware processor, perform amicro-analysis comprising dividing a digital image into a plurality ofblocks, for two or more of the plurality of blocks, determining a scorebased on a spatial frequency of the block, and generating a score mapfor the digital image based on the score for each of the two or moreblocks, and perform a macro-analysis comprising detecting artifacts inthe digital image, computing a degradation score based on detectedartifacts, and computing a whole-slide-quality score based on the scoremap and the degradation score.